Server Admin Log

From Wikitech

2024-05-01

  • 14:22 marostegui@cumin1002: dbctl commit (dc=all): 'db2154 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P61616 and previous config saved to /var/cache/conftool/dbconfig/20240501-142233-root.json
  • 14:20 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db1174', diff saved to https://phabricator.wikimedia.org/P61615 and previous config saved to /var/cache/conftool/dbconfig/20240501-142053-marostegui.json
  • 14:12 bking@deploy1002: helmfile [eqiad] DONE helmfile.d/services/rdf-streaming-updater: apply
  • 14:11 bking@deploy1002: helmfile [eqiad] START helmfile.d/services/rdf-streaming-updater: apply
  • 14:10 btullis@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cephosd1002.eqiad.wmnet with reason: host reimage
  • 14:08 btullis@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on cephosd1002.eqiad.wmnet with reason: host reimage
  • 14:07 marostegui@cumin1002: dbctl commit (dc=all): 'db2154 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P61614 and previous config saved to /var/cache/conftool/dbconfig/20240501-140728-root.json
  • 14:05 sukhe@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on dns7001.wikimedia.org with reason: host reimage
  • 14:05 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db1174 (T361627)', diff saved to https://phabricator.wikimedia.org/P61613 and previous config saved to /var/cache/conftool/dbconfig/20240501-140545-marostegui.json
  • 14:03 bking@deploy1002: helmfile [codfw] DONE helmfile.d/services/rdf-streaming-updater: apply
  • 14:03 marostegui@cumin1002: dbctl commit (dc=all): 'Depooling db1174 (T361627)', diff saved to https://phabricator.wikimedia.org/P61612 and previous config saved to /var/cache/conftool/dbconfig/20240501-140333-marostegui.json
  • 14:03 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1174.eqiad.wmnet with reason: Maintenance
  • 14:03 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db1174.eqiad.wmnet with reason: Maintenance
  • 14:03 sukhe@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on dns7001.wikimedia.org with reason: host reimage
  • 14:03 bking@deploy1002: helmfile [codfw] START helmfile.d/services/rdf-streaming-updater: apply
  • 13:59 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1171.eqiad.wmnet with reason: Maintenance
  • 13:59 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db1171.eqiad.wmnet with reason: Maintenance
  • 13:59 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db1170 (T361627)', diff saved to https://phabricator.wikimedia.org/P61611 and previous config saved to /var/cache/conftool/dbconfig/20240501-135915-marostegui.json
  • 13:52 marostegui@cumin1002: dbctl commit (dc=all): 'db2154 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P61610 and previous config saved to /var/cache/conftool/dbconfig/20240501-135222-root.json
  • 13:47 btullis@cumin1002: START - Cookbook sre.hosts.reimage for host cephosd1002.eqiad.wmnet with OS bookworm
  • 13:44 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db1170', diff saved to https://phabricator.wikimedia.org/P61609 and previous config saved to /var/cache/conftool/dbconfig/20240501-134407-marostegui.json
  • 13:37 marostegui@cumin1002: dbctl commit (dc=all): 'db2154 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P61608 and previous config saved to /var/cache/conftool/dbconfig/20240501-133717-root.json
  • 13:33 Amir1: promoting HNowlan (WMF) to admin in testwiki
  • 13:29 sukhe@cumin1002: START - Cookbook sre.hosts.reimage for host dns7001.wikimedia.org with OS bookworm
  • 13:29 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db1170', diff saved to https://phabricator.wikimedia.org/P61607 and previous config saved to /var/cache/conftool/dbconfig/20240501-132900-marostegui.json
  • 13:25 sukhe: running authdns-update for CR 1026119: depool magru text*
  • 13:22 marostegui@cumin1002: dbctl commit (dc=all): 'db2154 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P61606 and previous config saved to /var/cache/conftool/dbconfig/20240501-132211-root.json
  • 13:15 btullis@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host cephosd1001.eqiad.wmnet with OS bookworm
  • 13:13 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db1170 (T361627)', diff saved to https://phabricator.wikimedia.org/P61605 and previous config saved to /var/cache/conftool/dbconfig/20240501-131351-marostegui.json
  • 13:08 marostegui@cumin1002: dbctl commit (dc=all): 'Depooling db1170 (T361627)', diff saved to https://phabricator.wikimedia.org/P61604 and previous config saved to /var/cache/conftool/dbconfig/20240501-130822-marostegui.json
  • 13:08 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1170.eqiad.wmnet with reason: Maintenance
  • 13:07 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db1170.eqiad.wmnet with reason: Maintenance
  • 13:07 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db1158 (T361627)', diff saved to https://phabricator.wikimedia.org/P61603 and previous config saved to /var/cache/conftool/dbconfig/20240501-130747-marostegui.json
  • 13:07 marostegui@cumin1002: dbctl commit (dc=all): 'db2154 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P61602 and previous config saved to /var/cache/conftool/dbconfig/20240501-130704-root.json
  • 12:55 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2154.codfw.wmnet with OS bookworm
  • 12:52 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db1158', diff saved to https://phabricator.wikimedia.org/P61601 and previous config saved to /var/cache/conftool/dbconfig/20240501-125239-marostegui.json
  • 12:51 marostegui@cumin1002: dbctl commit (dc=all): 'db2154 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P61600 and previous config saved to /var/cache/conftool/dbconfig/20240501-125158-root.json
  • 12:48 btullis@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on cephosd1001.eqiad.wmnet with reason: host reimage
  • 12:45 btullis@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on cephosd1001.eqiad.wmnet with reason: host reimage
  • 12:24 btullis@cumin1002: START - Cookbook sre.hosts.reimage for host cephosd1001.eqiad.wmnet with OS bookworm
  • 12:22 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db1158 (T361627)', diff saved to https://phabricator.wikimedia.org/P61598 and previous config saved to /var/cache/conftool/dbconfig/20240501-122224-marostegui.json
  • 12:20 marostegui@cumin1002: dbctl commit (dc=all): 'Depooling db1158 (T361627)', diff saved to https://phabricator.wikimedia.org/P61597 and previous config saved to /var/cache/conftool/dbconfig/20240501-122012-marostegui.json
  • 12:20 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance
  • 12:19 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 8:00:00 on clouddb[1014,1018,1021].eqiad.wmnet,db1155.eqiad.wmnet with reason: Maintenance
  • 12:19 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db1158.eqiad.wmnet with reason: Maintenance
  • 12:19 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db1158.eqiad.wmnet with reason: Maintenance
  • 12:15 marostegui@cumin1002: START - Cookbook sre.hosts.reimage for host db2154.codfw.wmnet with OS bookworm
  • 12:15 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2218.codfw.wmnet with reason: Maintenance
  • 12:15 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db2218.codfw.wmnet with reason: Maintenance
  • 12:13 marostegui@cumin1002: dbctl commit (dc=all): 'Depool db2154', diff saved to https://phabricator.wikimedia.org/P61596 and previous config saved to /var/cache/conftool/dbconfig/20240501-121347-root.json
  • 12:08 marostegui@cumin1002: dbctl commit (dc=all): 'db2163 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P61595 and previous config saved to /var/cache/conftool/dbconfig/20240501-120833-root.json
  • 11:59 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2220 (T361627)', diff saved to https://phabricator.wikimedia.org/P61594 and previous config saved to /var/cache/conftool/dbconfig/20240501-115915-marostegui.json
  • 11:53 marostegui@cumin1002: dbctl commit (dc=all): 'db2163 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P61593 and previous config saved to /var/cache/conftool/dbconfig/20240501-115327-root.json
  • 11:44 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2220', diff saved to https://phabricator.wikimedia.org/P61592 and previous config saved to /var/cache/conftool/dbconfig/20240501-114408-marostegui.json
  • 11:38 marostegui@cumin1002: dbctl commit (dc=all): 'db2163 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P61591 and previous config saved to /var/cache/conftool/dbconfig/20240501-113821-root.json
  • 11:29 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2220', diff saved to https://phabricator.wikimedia.org/P61590 and previous config saved to /var/cache/conftool/dbconfig/20240501-112900-marostegui.json
  • 11:24 sukhe@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lvs7003.magru.wmnet with OS bullseye
  • 11:24 sukhe@cumin1002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - sukhe@cumin1002"
  • 11:23 marostegui@cumin1002: dbctl commit (dc=all): 'db2163 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P61589 and previous config saved to /var/cache/conftool/dbconfig/20240501-112315-root.json
  • 11:22 sukhe@cumin1002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - sukhe@cumin1002"
  • 11:17 sukhe@cumin1002: END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) for host lvs7002.magru.wmnet
  • 11:13 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2220 (T361627)', diff saved to https://phabricator.wikimedia.org/P61588 and previous config saved to /var/cache/conftool/dbconfig/20240501-111353-marostegui.json
  • 11:08 marostegui@cumin1002: dbctl commit (dc=all): 'Depooling db2220 (T361627)', diff saved to https://phabricator.wikimedia.org/P61587 and previous config saved to /var/cache/conftool/dbconfig/20240501-110834-marostegui.json
  • 11:08 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2220.codfw.wmnet with reason: Maintenance
  • 11:08 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db2220.codfw.wmnet with reason: Maintenance
  • 11:08 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2208 (T361627)', diff saved to https://phabricator.wikimedia.org/P61586 and previous config saved to /var/cache/conftool/dbconfig/20240501-110822-marostegui.json
  • 11:08 marostegui@cumin1002: dbctl commit (dc=all): 'db2163 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P61585 and previous config saved to /var/cache/conftool/dbconfig/20240501-110809-root.json
  • 11:07 sukhe@cumin1002: END (FAIL) - Cookbook sre.hosts.reboot-single (exit_code=1) for host lvs7001.magru.wmnet
  • 11:05 sukhe@cumin1002: START - Cookbook sre.hosts.reboot-single for host lvs7002.magru.wmnet
  • 10:58 sukhe@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on lvs7003.magru.wmnet with reason: host reimage
  • 10:55 sukhe@cumin1002: START - Cookbook sre.hosts.reboot-single for host lvs7001.magru.wmnet
  • 10:55 sukhe@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on lvs7003.magru.wmnet with reason: host reimage
  • 10:53 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2208', diff saved to https://phabricator.wikimedia.org/P61584 and previous config saved to /var/cache/conftool/dbconfig/20240501-105315-marostegui.json
  • 10:53 marostegui@cumin1002: dbctl commit (dc=all): 'db2163 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P61583 and previous config saved to /var/cache/conftool/dbconfig/20240501-105304-root.json
  • 10:42 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2163.codfw.wmnet with OS bookworm
  • 10:38 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2208', diff saved to https://phabricator.wikimedia.org/P61582 and previous config saved to /var/cache/conftool/dbconfig/20240501-103801-marostegui.json
  • 10:37 marostegui@cumin1002: dbctl commit (dc=all): 'db2163 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P61581 and previous config saved to /var/cache/conftool/dbconfig/20240501-103758-root.json
  • 10:33 arnaudb@cumin1002: dbctl commit (dc=all): 'db1157 (re)pooling @ 100%: post schema change repool', diff saved to https://phabricator.wikimedia.org/P61580 and previous config saved to /var/cache/conftool/dbconfig/20240501-103338-arnaudb.json
  • 10:30 sukhe@cumin1002: START - Cookbook sre.hosts.reimage for host lvs7003.magru.wmnet with OS bullseye
  • 10:30 sukhe@cumin1002: END (ERROR) - Cookbook sre.hosts.reimage (exit_code=97) for host lvs7003.magru.wmnet with OS bullseye
  • 10:29 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on db1246.eqiad.wmnet with reason: Down with HW issues
  • 10:29 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on db1246.eqiad.wmnet with reason: Down with HW issues
  • 10:28 sukhe@cumin1002: END (FAIL) - Cookbook sre.hardware.upgrade-firmware (exit_code=99) upgrade firmware for hosts ['lvs7003.magru.wmnet']
  • 10:27 sukhe@cumin1002: START - Cookbook sre.hardware.upgrade-firmware upgrade firmware for hosts ['lvs7003.magru.wmnet']
  • 10:22 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2208 (T361627)', diff saved to https://phabricator.wikimedia.org/P61579 and previous config saved to /var/cache/conftool/dbconfig/20240501-102253-marostegui.json
  • 10:22 sukhe@cumin1002: START - Cookbook sre.hosts.reimage for host lvs7003.magru.wmnet with OS bullseye
  • 10:20 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2163.codfw.wmnet with reason: host reimage
  • 10:18 arnaudb@cumin1002: dbctl commit (dc=all): 'db1157 (re)pooling @ 75%: post schema change repool', diff saved to https://phabricator.wikimedia.org/P61578 and previous config saved to /var/cache/conftool/dbconfig/20240501-101832-arnaudb.json
  • 10:17 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on db2163.codfw.wmnet with reason: host reimage
  • 10:17 marostegui@cumin1002: dbctl commit (dc=all): 'Depooling db2208 (T361627)', diff saved to https://phabricator.wikimedia.org/P61577 and previous config saved to /var/cache/conftool/dbconfig/20240501-101728-marostegui.json
  • 10:17 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2208.codfw.wmnet with reason: Maintenance
  • 10:17 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db2208.codfw.wmnet with reason: Maintenance
  • 10:12 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2200.codfw.wmnet with reason: Maintenance
  • 10:12 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db2200.codfw.wmnet with reason: Maintenance
  • 10:11 marostegui@cumin1002: dbctl commit (dc=all): 'db2164 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P61576 and previous config saved to /var/cache/conftool/dbconfig/20240501-101151-root.json
  • 10:07 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2198.codfw.wmnet with reason: Maintenance
  • 10:06 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db2198.codfw.wmnet with reason: Maintenance
  • 10:06 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2182 (T361627)', diff saved to https://phabricator.wikimedia.org/P61575 and previous config saved to /var/cache/conftool/dbconfig/20240501-100650-marostegui.json
  • 10:03 arnaudb@cumin1002: dbctl commit (dc=all): 'db1157 (re)pooling @ 50%: post schema change repool', diff saved to https://phabricator.wikimedia.org/P61574 and previous config saved to /var/cache/conftool/dbconfig/20240501-100326-arnaudb.json
  • 10:00 marostegui@cumin1002: START - Cookbook sre.hosts.reimage for host db2163.codfw.wmnet with OS bookworm
  • 09:58 marostegui@cumin1002: dbctl commit (dc=all): 'Depool db2163', diff saved to https://phabricator.wikimedia.org/P61573 and previous config saved to /var/cache/conftool/dbconfig/20240501-095845-root.json
  • 09:56 marostegui@cumin1002: dbctl commit (dc=all): 'db2164 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P61572 and previous config saved to /var/cache/conftool/dbconfig/20240501-095646-root.json
  • 09:52 topranks: restarting routinator service on rpki1001
  • 09:51 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2182', diff saved to https://phabricator.wikimedia.org/P61571 and previous config saved to /var/cache/conftool/dbconfig/20240501-095142-marostegui.json
  • 09:48 arnaudb@cumin1002: dbctl commit (dc=all): 'db1157 (re)pooling @ 25%: post schema change repool', diff saved to https://phabricator.wikimedia.org/P61570 and previous config saved to /var/cache/conftool/dbconfig/20240501-094821-arnaudb.json
  • 09:42 marostegui@deploy1002: Finished scap: Backport for etcd.php: Add es7 (T355285 T355424) (duration: 14m 53s)
  • 09:41 marostegui@cumin1002: dbctl commit (dc=all): 'db2164 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P61569 and previous config saved to /var/cache/conftool/dbconfig/20240501-094140-root.json
  • 09:36 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2182', diff saved to https://phabricator.wikimedia.org/P61568 and previous config saved to /var/cache/conftool/dbconfig/20240501-093635-marostegui.json
  • 09:33 arnaudb@cumin1002: dbctl commit (dc=all): 'db1157 (re)pooling @ 15%: post schema change repool', diff saved to https://phabricator.wikimedia.org/P61567 and previous config saved to /var/cache/conftool/dbconfig/20240501-093315-arnaudb.json
  • 09:30 marostegui@deploy1002: marostegui: Continuing with sync
  • 09:30 marostegui@deploy1002: marostegui: Backport for etcd.php: Add es7 (T355285 T355424) synced to the testservers (https://wikitech.wikimedia.org/wiki/Mwdebug)
  • 09:27 marostegui@deploy1002: Started scap: Backport for etcd.php: Add es7 (T355285 T355424)
  • 09:26 marostegui@cumin1002: dbctl commit (dc=all): 'db2164 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P61566 and previous config saved to /var/cache/conftool/dbconfig/20240501-092634-root.json
  • 09:22 topranks: withdrawing public prefix announcement to AS7195 to test backup in magru (T362421)
  • 09:21 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2182 (T361627)', diff saved to https://phabricator.wikimedia.org/P61565 and previous config saved to /var/cache/conftool/dbconfig/20240501-092125-marostegui.json
  • 09:18 arnaudb@cumin1002: dbctl commit (dc=all): 'db1157 (re)pooling @ 10%: post schema change repool', diff saved to https://phabricator.wikimedia.org/P61564 and previous config saved to /var/cache/conftool/dbconfig/20240501-091809-arnaudb.json
  • 09:15 marostegui@cumin1002: dbctl commit (dc=all): 'Depooling db2182 (T361627)', diff saved to https://phabricator.wikimedia.org/P61563 and previous config saved to /var/cache/conftool/dbconfig/20240501-091513-marostegui.json
  • 09:15 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2182.codfw.wmnet with reason: Maintenance
  • 09:14 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db2182.codfw.wmnet with reason: Maintenance
  • 09:14 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2168 (T361627)', diff saved to https://phabricator.wikimedia.org/P61562 and previous config saved to /var/cache/conftool/dbconfig/20240501-091451-marostegui.json
  • 09:13 marostegui@cumin1002: dbctl commit (dc=all): 'Push es7 codfw config T355424', diff saved to https://phabricator.wikimedia.org/P61561 and previous config saved to /var/cache/conftool/dbconfig/20240501-091352-marostegui.json
  • 09:11 marostegui@cumin1002: dbctl commit (dc=all): 'db2164 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P61560 and previous config saved to /var/cache/conftool/dbconfig/20240501-091128-root.json
  • 09:03 arnaudb@cumin1002: dbctl commit (dc=all): 'db1157 (re)pooling @ 5%: post schema change repool', diff saved to https://phabricator.wikimedia.org/P61559 and previous config saved to /var/cache/conftool/dbconfig/20240501-090303-arnaudb.json
  • 08:59 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2168', diff saved to https://phabricator.wikimedia.org/P61558 and previous config saved to /var/cache/conftool/dbconfig/20240501-085943-marostegui.json
  • 08:56 marostegui@cumin1002: dbctl commit (dc=all): 'db2164 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P61557 and previous config saved to /var/cache/conftool/dbconfig/20240501-085622-root.json
  • 08:52 marostegui@cumin1002: dbctl commit (dc=all): 'db1186 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P61556 and previous config saved to /var/cache/conftool/dbconfig/20240501-085223-root.json
  • 08:44 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2168', diff saved to https://phabricator.wikimedia.org/P61555 and previous config saved to /var/cache/conftool/dbconfig/20240501-084436-marostegui.json
  • 08:44 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2164.codfw.wmnet with OS bookworm
  • 08:41 marostegui@cumin1002: dbctl commit (dc=all): 'db2164 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P61554 and previous config saved to /var/cache/conftool/dbconfig/20240501-084116-root.json
  • 08:37 marostegui@cumin1002: dbctl commit (dc=all): 'db1186 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P61553 and previous config saved to /var/cache/conftool/dbconfig/20240501-083717-root.json
  • 08:36 marostegui@cumin1002: dbctl commit (dc=all): 'db1234 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P61552 and previous config saved to /var/cache/conftool/dbconfig/20240501-083641-root.json
  • 08:31 marostegui@cumin1002: dbctl commit (dc=all): 'Push es7 eqiad config T355285', diff saved to https://phabricator.wikimedia.org/P61551 and previous config saved to /var/cache/conftool/dbconfig/20240501-083120-marostegui.json
  • 08:29 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2168 (T361627)', diff saved to https://phabricator.wikimedia.org/P61550 and previous config saved to /var/cache/conftool/dbconfig/20240501-082928-marostegui.json
  • 08:23 marostegui@cumin1002: dbctl commit (dc=all): 'Depooling db2168 (T361627)', diff saved to https://phabricator.wikimedia.org/P61549 and previous config saved to /var/cache/conftool/dbconfig/20240501-082357-marostegui.json
  • 08:23 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2168.codfw.wmnet with reason: Maintenance
  • 08:23 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db2168.codfw.wmnet with reason: Maintenance
  • 08:23 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2159 (T361627)', diff saved to https://phabricator.wikimedia.org/P61548 and previous config saved to /var/cache/conftool/dbconfig/20240501-082334-marostegui.json
  • 08:22 marostegui@cumin1002: dbctl commit (dc=all): 'db1186 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P61547 and previous config saved to /var/cache/conftool/dbconfig/20240501-082211-root.json
  • 08:21 marostegui@cumin1002: dbctl commit (dc=all): 'db1234 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P61546 and previous config saved to /var/cache/conftool/dbconfig/20240501-082135-root.json
  • 08:20 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2164.codfw.wmnet with reason: host reimage
  • 08:17 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on db2164.codfw.wmnet with reason: host reimage
  • 08:08 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2159', diff saved to https://phabricator.wikimedia.org/P61545 and previous config saved to /var/cache/conftool/dbconfig/20240501-080827-marostegui.json
  • 08:07 marostegui@cumin1002: dbctl commit (dc=all): 'db1186 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P61544 and previous config saved to /var/cache/conftool/dbconfig/20240501-080706-root.json
  • 08:06 marostegui@cumin1002: dbctl commit (dc=all): 'db1234 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P61543 and previous config saved to /var/cache/conftool/dbconfig/20240501-080630-root.json
  • 08:05 arnaudb@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 12:00:00 on db1157.eqiad.wmnet with reason: Maintenance
  • 08:05 arnaudb@cumin1002: START - Cookbook sre.hosts.downtime for 12:00:00 on db1157.eqiad.wmnet with reason: Maintenance
  • 08:03 marostegui@cumin1002: dbctl commit (dc=all): 'db2166 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P61542 and previous config saved to /var/cache/conftool/dbconfig/20240501-080354-root.json
  • 07:59 marostegui@cumin1002: START - Cookbook sre.hosts.reimage for host db2164.codfw.wmnet with OS bookworm
  • 07:56 marostegui@cumin1002: dbctl commit (dc=all): 'Depool db2164', diff saved to https://phabricator.wikimedia.org/P61541 and previous config saved to /var/cache/conftool/dbconfig/20240501-075614-root.json
  • 07:53 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2159', diff saved to https://phabricator.wikimedia.org/P61540 and previous config saved to /var/cache/conftool/dbconfig/20240501-075320-marostegui.json
  • 07:52 marostegui@cumin1002: dbctl commit (dc=all): 'db1186 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P61539 and previous config saved to /var/cache/conftool/dbconfig/20240501-075200-root.json
  • 07:51 marostegui@cumin1002: dbctl commit (dc=all): 'db1234 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P61538 and previous config saved to /var/cache/conftool/dbconfig/20240501-075124-root.json
  • 07:48 marostegui@cumin1002: dbctl commit (dc=all): 'db2166 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P61537 and previous config saved to /var/cache/conftool/dbconfig/20240501-074848-root.json
  • 07:38 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2159 (T361627)', diff saved to https://phabricator.wikimedia.org/P61536 and previous config saved to /var/cache/conftool/dbconfig/20240501-073812-marostegui.json
  • 07:36 marostegui@cumin1002: dbctl commit (dc=all): 'db1186 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P61535 and previous config saved to /var/cache/conftool/dbconfig/20240501-073655-root.json
  • 07:36 marostegui@cumin1002: dbctl commit (dc=all): 'db1234 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P61534 and previous config saved to /var/cache/conftool/dbconfig/20240501-073615-root.json
  • 07:33 marostegui@cumin1002: dbctl commit (dc=all): 'db2166 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P61533 and previous config saved to /var/cache/conftool/dbconfig/20240501-073342-root.json
  • 07:32 marostegui@cumin1002: dbctl commit (dc=all): 'Depooling db2159 (T361627)', diff saved to https://phabricator.wikimedia.org/P61532 and previous config saved to /var/cache/conftool/dbconfig/20240501-073201-marostegui.json
  • 07:31 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 8:00:00 on db2187.codfw.wmnet with reason: Maintenance
  • 07:31 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 8:00:00 on db2187.codfw.wmnet with reason: Maintenance
  • 07:31 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2159.codfw.wmnet with reason: Maintenance
  • 07:31 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db2159.codfw.wmnet with reason: Maintenance
  • 07:31 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2150 (T361627)', diff saved to https://phabricator.wikimedia.org/P61531 and previous config saved to /var/cache/conftool/dbconfig/20240501-073123-marostegui.json
  • 07:21 marostegui@cumin1002: dbctl commit (dc=all): 'db1186 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P61530 and previous config saved to /var/cache/conftool/dbconfig/20240501-072149-root.json
  • 07:21 marostegui@cumin1002: dbctl commit (dc=all): 'db1234 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P61529 and previous config saved to /var/cache/conftool/dbconfig/20240501-072110-root.json
  • 07:18 marostegui@cumin1002: dbctl commit (dc=all): 'db2166 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P61528 and previous config saved to /var/cache/conftool/dbconfig/20240501-071836-root.json
  • 07:16 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2150', diff saved to https://phabricator.wikimedia.org/P61527 and previous config saved to /var/cache/conftool/dbconfig/20240501-071615-marostegui.json
  • 07:06 marostegui@cumin1002: dbctl commit (dc=all): 'db1234 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P61526 and previous config saved to /var/cache/conftool/dbconfig/20240501-070603-root.json
  • 07:03 marostegui@cumin1002: dbctl commit (dc=all): 'db2166 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P61525 and previous config saved to /var/cache/conftool/dbconfig/20240501-070330-root.json
  • 07:02 marostegui@cumin1002: END (PASS) - Cookbook sre.mysql.clone (exit_code=0) of db1186.eqiad.wmnet onto db1234.eqiad.wmnet
  • 07:01 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2150', diff saved to https://phabricator.wikimedia.org/P61524 and previous config saved to /var/cache/conftool/dbconfig/20240501-070108-marostegui.json
  • 06:58 marostegui@cumin1002: dbctl commit (dc=all): 'db1236 (re)pooling @ 100%: Repooling', diff saved to https://phabricator.wikimedia.org/P61523 and previous config saved to /var/cache/conftool/dbconfig/20240501-065845-root.json
  • 06:48 marostegui@cumin1002: dbctl commit (dc=all): 'db2166 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P61522 and previous config saved to /var/cache/conftool/dbconfig/20240501-064824-root.json
  • 06:46 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2150 (T361627)', diff saved to https://phabricator.wikimedia.org/P61521 and previous config saved to /var/cache/conftool/dbconfig/20240501-064600-marostegui.json
  • 06:43 marostegui@cumin1002: dbctl commit (dc=all): 'db1236 (re)pooling @ 75%: Repooling', diff saved to https://phabricator.wikimedia.org/P61520 and previous config saved to /var/cache/conftool/dbconfig/20240501-064339-root.json
  • 06:39 marostegui@cumin1002: dbctl commit (dc=all): 'Depooling db2150 (T361627)', diff saved to https://phabricator.wikimedia.org/P61519 and previous config saved to /var/cache/conftool/dbconfig/20240501-063942-marostegui.json
  • 06:39 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2150.codfw.wmnet with reason: Maintenance
  • 06:39 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db2150.codfw.wmnet with reason: Maintenance
  • 06:39 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2122 (T361627)', diff saved to https://phabricator.wikimedia.org/P61518 and previous config saved to /var/cache/conftool/dbconfig/20240501-063919-marostegui.json
  • 06:38 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db2166.codfw.wmnet with OS bookworm
  • 06:33 marostegui@cumin1002: dbctl commit (dc=all): 'db2166 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P61517 and previous config saved to /var/cache/conftool/dbconfig/20240501-063318-root.json
  • 06:28 marostegui@cumin1002: dbctl commit (dc=all): 'db1236 (re)pooling @ 50%: Repooling', diff saved to https://phabricator.wikimedia.org/P61516 and previous config saved to /var/cache/conftool/dbconfig/20240501-062833-root.json
  • 06:24 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2122', diff saved to https://phabricator.wikimedia.org/P61515 and previous config saved to /var/cache/conftool/dbconfig/20240501-062407-marostegui.json
  • 06:17 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db2166.codfw.wmnet with reason: host reimage
  • 06:15 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on db2166.codfw.wmnet with reason: host reimage
  • 06:13 marostegui@cumin1002: dbctl commit (dc=all): 'db1236 (re)pooling @ 25%: Repooling', diff saved to https://phabricator.wikimedia.org/P61514 and previous config saved to /var/cache/conftool/dbconfig/20240501-061327-root.json
  • 06:09 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2122', diff saved to https://phabricator.wikimedia.org/P61513 and previous config saved to /var/cache/conftool/dbconfig/20240501-060900-marostegui.json
  • 05:58 marostegui@cumin1002: dbctl commit (dc=all): 'db1236 (re)pooling @ 10%: Repooling', diff saved to https://phabricator.wikimedia.org/P61512 and previous config saved to /var/cache/conftool/dbconfig/20240501-055822-root.json
  • 05:58 marostegui@cumin1002: START - Cookbook sre.hosts.reimage for host db2166.codfw.wmnet with OS bookworm
  • 05:56 marostegui@cumin1002: dbctl commit (dc=all): 'Depool db2166', diff saved to https://phabricator.wikimedia.org/P61511 and previous config saved to /var/cache/conftool/dbconfig/20240501-055657-root.json
  • 05:53 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2122 (T361627)', diff saved to https://phabricator.wikimedia.org/P61510 and previous config saved to /var/cache/conftool/dbconfig/20240501-055353-marostegui.json
  • 05:47 marostegui@cumin1002: dbctl commit (dc=all): 'Depooling db2122 (T361627)', diff saved to https://phabricator.wikimedia.org/P61509 and previous config saved to /var/cache/conftool/dbconfig/20240501-054720-marostegui.json
  • 05:47 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2122.codfw.wmnet with reason: Maintenance
  • 05:47 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db2122.codfw.wmnet with reason: Maintenance
  • 05:46 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2121 (T361627)', diff saved to https://phabricator.wikimedia.org/P61508 and previous config saved to /var/cache/conftool/dbconfig/20240501-054657-marostegui.json
  • 05:43 marostegui@cumin1002: dbctl commit (dc=all): 'db1236 (re)pooling @ 5%: Repooling', diff saved to https://phabricator.wikimedia.org/P61507 and previous config saved to /var/cache/conftool/dbconfig/20240501-054316-root.json
  • 05:34 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on es[1035,1039-1040].eqiad.wmnet with reason: Setting up T355285 T355424
  • 05:33 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on es[1035,1039-1040].eqiad.wmnet with reason: Setting up T355285 T355424
  • 05:33 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on 6 hosts with reason: Setting up T355285 T355424
  • 05:33 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on 6 hosts with reason: Setting up T355285 T355424
  • 05:31 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2121', diff saved to https://phabricator.wikimedia.org/P61506 and previous config saved to /var/cache/conftool/dbconfig/20240501-053149-marostegui.json
  • 05:31 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1234.eqiad.wmnet with OS bookworm
  • 05:29 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host db1236.eqiad.wmnet with OS bookworm
  • 05:28 marostegui@cumin1002: dbctl commit (dc=all): 'db1236 (re)pooling @ 1%: Repooling', diff saved to https://phabricator.wikimedia.org/P61505 and previous config saved to /var/cache/conftool/dbconfig/20240501-052810-root.json
  • 05:23 marostegui@cumin1002: START - Cookbook sre.mysql.clone of db1186.eqiad.wmnet onto db1234.eqiad.wmnet
  • 05:18 marostegui@cumin1002: dbctl commit (dc=all): 'Depool db1186 to clone db1234 T363890', diff saved to https://phabricator.wikimedia.org/P61504 and previous config saved to /var/cache/conftool/dbconfig/20240501-051848-marostegui.json
  • 05:16 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2121', diff saved to https://phabricator.wikimedia.org/P61503 and previous config saved to /var/cache/conftool/dbconfig/20240501-051642-marostegui.json
  • 05:14 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1236.eqiad.wmnet with reason: host reimage
  • 05:10 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on db1236.eqiad.wmnet with reason: host reimage
  • 05:10 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on db1234.eqiad.wmnet with reason: host reimage
  • 05:08 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 7 days, 0:00:00 on db1246.eqiad.wmnet with reason: Down with HW issues
  • 05:08 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 7 days, 0:00:00 on db1246.eqiad.wmnet with reason: Down with HW issues
  • 05:07 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on db1234.eqiad.wmnet with reason: host reimage
  • 05:01 marostegui@cumin1002: dbctl commit (dc=all): 'Repooling after maintenance db2121 (T361627)', diff saved to https://phabricator.wikimedia.org/P61502 and previous config saved to /var/cache/conftool/dbconfig/20240501-050135-marostegui.json
  • 04:57 marostegui@cumin1002: START - Cookbook sre.hosts.reimage for host db1236.eqiad.wmnet with OS bookworm
  • 04:56 marostegui@cumin1002: dbctl commit (dc=all): 'Depool db1236', diff saved to https://phabricator.wikimedia.org/P61501 and previous config saved to /var/cache/conftool/dbconfig/20240501-045624-marostegui.json
  • 04:55 marostegui@cumin1002: dbctl commit (dc=all): 'Depooling db2121 (T361627)', diff saved to https://phabricator.wikimedia.org/P61500 and previous config saved to /var/cache/conftool/dbconfig/20240501-045517-marostegui.json
  • 04:55 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2121.codfw.wmnet with reason: Maintenance
  • 04:54 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db2121.codfw.wmnet with reason: Maintenance
  • 04:54 marostegui@cumin1002: START - Cookbook sre.hosts.reimage for host db1234.eqiad.wmnet with OS bookworm
  • 04:50 marostegui@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 4:00:00 on db2098.codfw.wmnet with reason: Maintenance
  • 04:50 marostegui@cumin1002: START - Cookbook sre.hosts.downtime for 4:00:00 on db2098.codfw.wmnet with reason: Maintenance
  • 02:31 sukhe@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lvs7002.magru.wmnet with OS bullseye
  • 02:31 sukhe@cumin1002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - sukhe@cumin1002"
  • 02:29 sukhe@cumin1002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - sukhe@cumin1002"
  • 02:07 sukhe@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on lvs7002.magru.wmnet with reason: host reimage
  • 02:04 sukhe@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on lvs7002.magru.wmnet with reason: host reimage
  • 01:37 sukhe@cumin1002: START - Cookbook sre.hosts.reimage for host lvs7002.magru.wmnet with OS bullseye
  • 01:26 sukhe@cumin1002: END (PASS) - Cookbook sre.hosts.reimage (exit_code=0) for host lvs7001.magru.wmnet with OS bullseye
  • 01:26 sukhe@cumin1002: END (PASS) - Cookbook sre.puppet.sync-netbox-hiera (exit_code=0) generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - sukhe@cumin1002"
  • 01:25 sukhe@cumin1002: START - Cookbook sre.puppet.sync-netbox-hiera generate netbox hiera data: "Triggered by cookbooks.sre.hosts.reimage: Host reimage - sukhe@cumin1002"
  • 01:02 sukhe@cumin1002: END (PASS) - Cookbook sre.hosts.downtime (exit_code=0) for 2:00:00 on lvs7001.magru.wmnet with reason: host reimage
  • 00:58 sukhe@cumin1002: START - Cookbook sre.hosts.downtime for 2:00:00 on lvs7001.magru.wmnet with reason: host reimage
  • 00:33 sukhe@cumin1002: START - Cookbook sre.hosts.reimage for host lvs7001.magru.wmnet with OS bullseye
  • 00:23 xcollazo@deploy1002: Finished deploy [airflow-dags/analytics@b10376a]: (no justification provided) (duration: 00m 31s)
  • 00:22 xcollazo@deploy1002: Started deploy [airflow-dags/analytics@b10376a]: (no justification provided)
  • 00:05 eileen: civicrm upgraded from 393e1deb to 3ac4043

Archives

See Server Admin Log/Archives.

Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Server_Admin_Log&oldid=2174046"